{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exporting ATT&CK Group Navigator Layers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Get Relationship STIX Objects - (Manual)\n",
    "-----------------------\n",
    "I believe it is important to understand the code behind the main functions available in the Python library [attackcti](https://attackcti.readthedocs.io/en/latest/index.html). I highly recommend to first read the docs I put together about [cti-taxii-client](https://attackcti.readthedocs.io/en/latest/taxii_client.html) and [cti-python-stix2](https://attackcti.readthedocs.io/en/latest/stix.html) libraries.\n",
    "Those two summarize several of the concepts that I had to read to understand how to perform a simple query against ATT&CK's TAXII server"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import STIX and TAXII Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from stix2 import TAXIICollectionSource, Filter, CompositeDataSource\n",
    "from taxii2client import Collection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set ATT&CK TAXII Collection ID Variables\n",
    "The public ATT&CK TAXII instance has three main collections (Enterprise, Pre and Mobile). Every collection has an ID which attackcti uses to retrieve ATT&CK STIX objects from all those matrices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "ATTCK_STIX_COLLECTIONS = \"https://cti-taxii.mitre.org/stix/collections/\"\n",
    "ENTERPRISE_ATTCK = \"95ecc380-afe9-11e4-9b6c-751b66dd541e\"\n",
    "PRE_ATTCK = \"062767bd-02d2-4b72-84ba-56caef0f8658\"\n",
    "MOBILE_ATTCK = \"2f669986-b40b-4423-b720-4396ca6a462b\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initialize TAXII Collection Sources\n",
    "According to [STIX2 docs](https://stix2.readthedocs.io/en/latest/index.html), the [TAXIICollectionSource API](https://stix2.readthedocs.io/en/latest/api/datastore/stix2.datastore.taxii.html#stix2.datastore.taxii.TAXIICollectionSource) provides an interface for searching/retrieving STIX objects from a local/remote TAXII Collection endpoint. In our case, we are pointing to our ATT&CK TAXII Collection instances (https://cti-taxii.mitre.org/stix/collections/<Collection ID>)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "ENTERPRISE_COLLECTION = Collection(ATTCK_STIX_COLLECTIONS + ENTERPRISE_ATTCK + \"/\")\n",
    "TC_ENTERPRISE_SOURCE = TAXIICollectionSource(ENTERPRISE_COLLECTION)\n",
    "PRE_COLLECTION = Collection(ATTCK_STIX_COLLECTIONS + PRE_ATTCK + \"/\")\n",
    "TC_PRE_SOURCE = TAXIICollectionSource(PRE_COLLECTION)\n",
    "MOBILE_COLLECTION = Collection(ATTCK_STIX_COLLECTIONS + MOBILE_ATTCK + \"/\")\n",
    "TC_MOBILE_SOURCE = TAXIICollectionSource(MOBILE_COLLECTION)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initialize a Composite Data Source\n",
    "According to [STIX2 docs](https://stix2.readthedocs.io/en/latest/index.html), a user can have a single [CompositeDataSource](https://stix2.readthedocs.io/en/latest/api/stix2.datastore.html#stix2.datastore.CompositeDataSource) as an interface to a set of DataSources. When an API call is made to the CompositeDataSource, it is delegated to each of the (real) DataSources that are attached to it. In our case, we have three TAXIICollection sources (Enterprise, PRE and Mobile) as defined in our previous step. Therefore, we can use the CompositeDataSource class and the add_data_sources method to attach every ATT&CK TAXIICollection source and be able to query all of them at the same time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "COMPOSITE_DS = CompositeDataSource()\n",
    "COMPOSITE_DS.add_data_sources([TC_ENTERPRISE_SOURCE, TC_PRE_SOURCE, TC_MOBILE_SOURCE])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Retrieve all relationships\n",
    "Now that we can query all the ATT&CK TAXIICollection sources at once, we can use the query method and a set of filters to retrieve STIX objects of type relationship"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Relationship(type='relationship', id='relationship--25b225e4-2365-4c00-8a27-f578a7fd94c8', created_by_ref='identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5', created='2019-10-23T14:26:33.079Z', modified='2019-10-23T14:26:33.079Z', relationship_type='mitigates', description='Consider rotating access keys within a certain number of days to reduce the effectiveness of stolen credentials.', source_ref='course-of-action--90c218c3-fbf8-4830-98a7-e8cfb7eaa485', target_ref='attack-pattern--d4bdbdea-eaec-4071-b4f9-5105e12ea4b6', object_marking_refs=['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rels = COMPOSITE_DS.query(Filter(\"type\", \"=\", \"relationship\"))\n",
    "rels[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Retrieve all relationships from an specific STIX object\n",
    "What if you want to be very specific and get relationships from a specific STIX objects? You can use the [relationships](https://stix2.readthedocs.io/en/latest/api/stix2.datastore.html#stix2.datastore.CompositeDataSource.relationships) method from the [CompositeDataSource](https://stix2.readthedocs.io/en/latest/api/stix2.datastore.html#stix2.datastore.CompositeDataSource) class to retrieve relationships involving a given STIX object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Relationship(type='relationship', id='relationship--4d1d7045-4492-492c-9522-2885d6bd96f6', created_by_ref='identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5', created='2019-09-24T13:01:20.471Z', modified='2019-09-24T13:01:20.472Z', relationship_type='uses', source_ref='intrusion-set--18854f55-ac7c-4634-bd9a-352dd07613b7', target_ref='malware--cfc75b0d-e579-40ae-ad07-a1ce00d49a6c', external_references=[ExternalReference(source_name='FireEye APT41 Aug 2019', description='Fraser, N., et al. (2019, August 7). Double DragonAPT41, a dual espionage and cyber crime operation APT41. Retrieved September 23, 2019.', url='https://content.fireeye.com/apt-41/rpt-apt41')], object_marking_refs=['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from attackcti import attack_client\n",
    "lift = attack_client()\n",
    "\n",
    "groups = lift.get_groups()\n",
    "groups = lift.remove_revoked(groups)\n",
    "\n",
    "rels = COMPOSITE_DS.relationships(groups[0], 'uses', source_only=True)\n",
    "rels[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Get Relationship STIX Objects - (Automatic)\n",
    "-----------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Retrieve all relationships"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from attackcti import attack_client\n",
    "lift = attack_client()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 3.51 s, sys: 270 ms, total: 3.78 s\n",
      "Wall time: 5.75 s\n"
     ]
    }
   ],
   "source": [
    "%time all_relationships = lift.get_relationships()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Relationship(type='relationship', id='relationship--25b225e4-2365-4c00-8a27-f578a7fd94c8', created_by_ref='identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5', created='2019-10-23T14:26:33.079Z', modified='2019-10-23T14:26:33.079Z', relationship_type='mitigates', description='Consider rotating access keys within a certain number of days to reduce the effectiveness of stolen credentials.', source_ref='course-of-action--90c218c3-fbf8-4830-98a7-e8cfb7eaa485', target_ref='attack-pattern--d4bdbdea-eaec-4071-b4f9-5105e12ea4b6', object_marking_refs=['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_relationships[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Retrieve all relationships from an specific STIX object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "groups = lift.get_groups()\n",
    "groups = lift.remove_revoked(groups)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 900 ms, sys: 60 ms, total: 960 ms\n",
      "Wall time: 2.76 s\n"
     ]
    }
   ],
   "source": [
    "%time group_relationships = lift.get_relationships_by_object(groups[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Relationship(type='relationship', id='relationship--4d1d7045-4492-492c-9522-2885d6bd96f6', created_by_ref='identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5', created='2019-09-24T13:01:20.471Z', modified='2019-09-24T13:01:20.472Z', relationship_type='uses', source_ref='intrusion-set--18854f55-ac7c-4634-bd9a-352dd07613b7', target_ref='malware--cfc75b0d-e579-40ae-ad07-a1ce00d49a6c', external_references=[ExternalReference(source_name='FireEye APT41 Aug 2019', description='Fraser, N., et al. (2019, August 7). Double DragonAPT41, a dual espionage and cyber crime operation APT41. Retrieved September 23, 2019.', url='https://content.fireeye.com/apt-41/rpt-apt41')], object_marking_refs=['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "group_relationships[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Retrive Techniques used by one Group - (Manual)\n",
    "-----------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case we want relationship objects that have target_ref values of type attack-pattern. Following the manual code I shared above, and the results from the `get_relationships_by_object()` function, you can simply query the ATT&CK Enterprise TAXIICollection source with the filter below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AttackPattern(type='attack-pattern', id='attack-pattern--cd25c1b4-935c-4f0e-ba8d-552f28bc4783', created_by_ref='identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5', created='2019-04-17T14:50:05.682Z', modified='2019-10-10T18:40:46.985Z', name='Resource Hijacking', description='Adversaries may leverage the resources of co-opted systems in order to solve resource intensive problems which may impact system and/or hosted service availability. \\n\\nOne common purpose for Resource Hijacking is to validate transactions of cryptocurrency networks and earn virtual currency. Adversaries may consume enough system resources to negatively impact and/or cause affected machines to become unresponsive.(Citation: Kaspersky Lazarus Under The Hood Blog 2017) Servers and cloud-based(Citation: CloudSploit - Unused AWS Regions) systems are common targets because of the high potential for available resources, but user endpoint systems may also be compromised and used for Resource Hijacking and cryptocurrency mining.', kill_chain_phases=[KillChainPhase(kill_chain_name='mitre-attack', phase_name='impact')], external_references=[ExternalReference(source_name='mitre-attack', url='https://attack.mitre.org/techniques/T1496', external_id='T1496'), ExternalReference(source_name='Kaspersky Lazarus Under The Hood Blog 2017', description='GReAT. (2017, April 3). Lazarus Under the Hood. Retrieved April 17, 2019.', url='https://securelist.com/lazarus-under-the-hood/77908/'), ExternalReference(source_name='CloudSploit - Unused AWS Regions', description='CloudSploit. (2019, June 8). The Danger of Unused AWS Regions. Retrieved October 8, 2019.', url='https://blog.cloudsploit.com/the-danger-of-unused-aws-regions-af0bf1b878fc')], object_marking_refs=['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'], x_mitre_data_sources=['Azure activity logs', 'Stackdriver logs', 'AWS CloudTrail logs', 'Process use of network', 'Process monitoring', 'Network protocol analysis', 'Network device logs'], x_mitre_detection='Consider monitoring process resource usage to determine anomalous activity associated with malicious hijacking of computer resources such as CPU, memory, and graphics processing resources. Monitor for suspicious use of network resources associated with cryptocurrency mining software. Monitor for common cryptomining software process names and files on local systems that may indicate compromise and resource usage.', x_mitre_impact_type=['Availability'], x_mitre_permissions_required=['User', 'Administrator'], x_mitre_platforms=['Linux', 'macOS', 'Windows', 'AWS', 'GCP', 'Azure'], x_mitre_version='1.1')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filter_objects = [\n",
    "  Filter('type', '=', 'attack-pattern'),\n",
    "  Filter('id', '=', [r.target_ref for r in group_relationships])\n",
    "]\n",
    "techniques_used = TC_ENTERPRISE_SOURCE.query(filter_objects)\n",
    "techniques_used[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Retrive Techniques used by one Group - (Automatic)\n",
    "-----------------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AttackPattern(type='attack-pattern', id='attack-pattern--cd25c1b4-935c-4f0e-ba8d-552f28bc4783', created_by_ref='identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5', created='2019-04-17T14:50:05.682Z', modified='2019-10-10T18:40:46.985Z', name='Resource Hijacking', description='Adversaries may leverage the resources of co-opted systems in order to solve resource intensive problems which may impact system and/or hosted service availability. \\n\\nOne common purpose for Resource Hijacking is to validate transactions of cryptocurrency networks and earn virtual currency. Adversaries may consume enough system resources to negatively impact and/or cause affected machines to become unresponsive.(Citation: Kaspersky Lazarus Under The Hood Blog 2017) Servers and cloud-based(Citation: CloudSploit - Unused AWS Regions) systems are common targets because of the high potential for available resources, but user endpoint systems may also be compromised and used for Resource Hijacking and cryptocurrency mining.', kill_chain_phases=[KillChainPhase(kill_chain_name='mitre-attack', phase_name='impact')], external_references=[ExternalReference(source_name='mitre-attack', url='https://attack.mitre.org/techniques/T1496', external_id='T1496'), ExternalReference(source_name='Kaspersky Lazarus Under The Hood Blog 2017', description='GReAT. (2017, April 3). Lazarus Under the Hood. Retrieved April 17, 2019.', url='https://securelist.com/lazarus-under-the-hood/77908/'), ExternalReference(source_name='CloudSploit - Unused AWS Regions', description='CloudSploit. (2019, June 8). The Danger of Unused AWS Regions. Retrieved October 8, 2019.', url='https://blog.cloudsploit.com/the-danger-of-unused-aws-regions-af0bf1b878fc')], object_marking_refs=['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'], x_mitre_data_sources=['Azure activity logs', 'Stackdriver logs', 'AWS CloudTrail logs', 'Process use of network', 'Process monitoring', 'Network protocol analysis', 'Network device logs'], x_mitre_detection='Consider monitoring process resource usage to determine anomalous activity associated with malicious hijacking of computer resources such as CPU, memory, and graphics processing resources. Monitor for suspicious use of network resources associated with cryptocurrency mining software. Monitor for common cryptomining software process names and files on local systems that may indicate compromise and resource usage.', x_mitre_impact_type=['Availability'], x_mitre_permissions_required=['User', 'Administrator'], x_mitre_platforms=['Linux', 'macOS', 'Windows', 'AWS', 'GCP', 'Azure'], x_mitre_version='1.1')"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from attackcti import attack_client\n",
    "lift = attack_client()\n",
    "groups = lift.get_groups()\n",
    "groups = lift.remove_revoked(groups)\n",
    "group_techniques = lift.get_techniques_used_by_group(groups[0])\n",
    "group_techniques[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Retrive all Techniques used by all Groups - (Manual)\n",
    "-----------------------\n",
    "You can apply the same get_techniques_used_by_group() function, but against all the groups STIX objects that the get_groups() function retrieves. You can do a simple for loop over more than 90 groups. However, it takes longer than what I would like it to take. Therefore, I decided to go a different route and started testing some code."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get all groups and techniques"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from attackcti import attack_client\n",
    "lift = attack_client()\n",
    "groups = lift.get_groups()\n",
    "techniques = lift.get_techniques()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Filter Group objects using techniques"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"type\": \"relationship\",\n",
      "    \"id\": \"relationship--0f880e99-efaa-4e85-91c3-cac3d81d6b9a\",\n",
      "    \"created_by_ref\": \"identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5\",\n",
      "    \"created\": \"2019-10-15T22:28:40.394Z\",\n",
      "    \"modified\": \"2019-10-15T22:28:40.394Z\",\n",
      "    \"relationship_type\": \"uses\",\n",
      "    \"description\": \"[Machete](https://attack.mitre.org/groups/G0095) has has relied on users opening malicious links or attachments delivered through spearphishing to execute malware.\",\n",
      "    \"source_ref\": \"intrusion-set--38863958-a201-4ce1-9dbe-539b0b6804e0\",\n",
      "    \"target_ref\": \"attack-pattern--8c32eb4d-805f-4fc5-bf60-c4d476c131b5\",\n",
      "    \"external_references\": [\n",
      "        {\n",
      "            \"source_name\": \"Cylance Machete Mar 2017\",\n",
      "            \"description\": \"The Cylance Threat Research Team. (2017, March 22). El Machete's Malware Attacks Cut Through LATAM. Retrieved September 13, 2019.\",\n",
      "            \"url\": \"https://threatvector.cylance.com/en_us/home/el-machete-malware-attacks-cut-through-latam.html\"\n",
      "        },\n",
      "        {\n",
      "            \"source_name\": \"Securelist Machete Aug 2014\",\n",
      "            \"description\": \"Kaspersky Global Research and Analysis Team. (2014, August 20). El Machete. Retrieved September 13, 2019.\",\n",
      "            \"url\": \"https://securelist.com/el-machete/66108/\"\n",
      "        },\n",
      "        {\n",
      "            \"source_name\": \"ESET Machete July 2019\",\n",
      "            \"description\": \"ESET. (2019, July). MACHETE JUST GOT SHARPER Venezuelan government institutions under attack. Retrieved September 13, 2019.\",\n",
      "            \"url\": \"https://www.welivesecurity.com/wp-content/uploads/2019/08/ESET_Machete.pdf\"\n",
      "        }\n",
      "    ],\n",
      "    \"object_marking_refs\": [\n",
      "        \"marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168\"\n",
      "    ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "from stix2.utils import get_type_from_id\n",
    "group_relationships = []\n",
    "for rel in all_relationships:\n",
    "    if get_type_from_id(rel.source_ref) == 'intrusion-set'\\\n",
    "    and get_type_from_id(rel.target_ref) == 'attack-pattern':\n",
    "        group_relationships.append(rel)\n",
    "len(group_relationships)\n",
    "print(group_relationships[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Match Group -> Relationships Intrusion-set ID\n",
    "Then, I just take all the group_relationships I got, and look for the specific `intrusion-set (Group)` id in the groups STIX objects. Once there is a match, I create new fields on the `intrusion-set (Group)` STIX object to add additional information about the `attack-pattern (Technique)` in the relationship object. The most important additional metadata is the target_ref field which points to the specific `attack-pattern (Technique)` id involving the `group`. The results are then added to a new list named `group_techniques_ref` ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "group_techniques_ref = []\n",
    "for g in groups:\n",
    "    for rel in group_relationships:\n",
    "        if g['id'] == rel['source_ref']:\n",
    "            gs = json.loads(g.serialize())\n",
    "            gs\n",
    "            gs['technique_ref'] = rel['target_ref']\n",
    "            gs['relationship_description'] = rel['description']\n",
    "            gs['relationship_id'] = rel['id']\n",
    "            group_techniques_ref.append(gs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Match Attack-patterns -> Intrusion-set object ID\n",
    "I apply the same concept as before, and just loop through all the attack-pattern objects and look for the specific attack-pattern id in the initial relationships STIX objects. Once there is a match, I add additional information from the attack-pattern itself to the python dictionaries in the `group_techniques_ref` list. The results then get added to a new list named `groups_use_techniques`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'aliases': ['APT41'],\n",
       " 'type': 'intrusion-set',\n",
       " 'name': 'APT41',\n",
       " 'description': '[APT41](https://attack.mitre.org/groups/G0096) is a group that carries out Chinese state-sponsored espionage activity in addition to financially motivated activity. [APT41](https://attack.mitre.org/groups/G0096) has been active since as early as 2012. The group has been observed targeting healthcare, telecom, technology, and video game industries in 14 countries.(Citation: FireEye APT41 Aug 2019)',\n",
       " 'external_references': [{'external_id': 'G0096',\n",
       "   'source_name': 'mitre-attack',\n",
       "   'url': 'https://attack.mitre.org/groups/G0096'},\n",
       "  {'description': '(Citation: FireEye APT41 2019)', 'source_name': 'APT41'},\n",
       "  {'description': 'Fraser, N., et al. (2019, August 7). Double DragonAPT41, a dual espionage and cyber crime operation APT41. Retrieved September 23, 2019.',\n",
       "   'source_name': 'FireEye APT41 Aug 2019',\n",
       "   'url': 'https://content.fireeye.com/apt-41/rpt-apt41'}],\n",
       " 'created_by_ref': 'identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5',\n",
       " 'created': '2019-09-23T13:43:36.945Z',\n",
       " 'id': 'intrusion-set--18854f55-ac7c-4634-bd9a-352dd07613b7',\n",
       " 'modified': '2019-10-14T21:52:59.301Z',\n",
       " 'object_marking_refs': ['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'],\n",
       " 'x_mitre_version': '1.0',\n",
       " 'technique_ref': 'attack-pattern--3489cfc5-640f-4bb3-a103-9137b97de79f',\n",
       " 'relationship_description': ' [APT41](https://attack.mitre.org/groups/G0096) used the <code>net share</code> command as part of network reconnaissance.',\n",
       " 'relationship_id': 'relationship--4f6e677d-427b-4342-b35c-57f4f3ad4ff8',\n",
       " 'technique': 'Network Share Discovery',\n",
       " 'technique_description': 'Networks often contain shared network drives and folders that enable users to access file directories on various systems across a network. \\n\\n### Windows\\n\\nFile sharing over a Windows network occurs over the SMB protocol. (Citation: Wikipedia Shared Resource) (Citation: TechNet Shared Folder)\\n\\n[Net](https://attack.mitre.org/software/S0039) can be used to query a remote system for available shared drives using the <code>net view \\\\\\\\remotesystem</code> command. It can also be used to query shared drives on the local system using <code>net share</code>.\\n\\nAdversaries may look for folders and drives shared on remote systems as a means of identifying sources of information to gather as a precursor for Collection and to identify potential systems of interest for Lateral Movement.\\n\\n### Mac\\n\\nOn Mac, locally mounted shares can be viewed with the <code>df -aH</code> command.\\n\\n### Cloud\\n\\nCloud virtual networks may contain remote network shares or file storage services accessible to an adversary after they have obtained access to a system. For example, AWS, GCP, and Azure support creation of Network File System (NFS) shares and Server Message Block (SMB) shares that may be mapped on endpoint or cloud-based systems.(Citation: Amazon Creating an NFS File Share)(Citation: Google File servers on Compute Engine)',\n",
       " 'tactic': ['discovery'],\n",
       " 'technique_id': 'T1135',\n",
       " 'matrix': 'mitre-attack',\n",
       " 'platform': ['macOS', 'Windows', 'AWS', 'GCP', 'Azure'],\n",
       " 'data_sources': ['Process monitoring',\n",
       "  'Process command-line parameters',\n",
       "  'Network protocol analysis',\n",
       "  'Process use of network'],\n",
       " 'permissions_required': ['User']}"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "groups_use_techniques = []\n",
    "for gt in group_techniques_ref:\n",
    "    for t in techniques:\n",
    "        if gt['technique_ref'] == t['id']:\n",
    "            tactic_list = list()\n",
    "            for phase in t['kill_chain_phases']:\n",
    "                tactic_list.append(phase['phase_name'])\n",
    "                gt['technique'] = t['name']\n",
    "                gt['technique_description'] = t['description']\n",
    "                gt['tactic'] = tactic_list\n",
    "                gt['technique_id'] = t['external_references'][0]['external_id']\n",
    "                gt['matrix'] =  t['external_references'][0]['source_name']\n",
    "                if 'x_mitre_platforms' in t.keys():\n",
    "                    gt['platform'] = t['x_mitre_platforms']\n",
    "                if 'x_mitre_data_sources' in t.keys():\n",
    "                    gt['data_sources'] = t['x_mitre_data_sources']\n",
    "                if 'x_mitre_permissions_required' in t.keys():\n",
    "                    gt['permissions_required'] = t['x_mitre_permissions_required']\n",
    "                if 'x_mitre_effective_permissions' in t.keys():\n",
    "                    gt['effective_permissions'] = t['x_mitre_effective_permissions']\n",
    "                groups_use_techniques.append(gt)\n",
    "groups_use_techniques[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Retrive all Techniques used by all Groups - (Automatic)\n",
    "-----------------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 7.4 s, sys: 570 ms, total: 7.97 s\n",
      "Wall time: 11.4 s\n"
     ]
    }
   ],
   "source": [
    "from attackcti import attack_client\n",
    "lift = attack_client()\n",
    "%time techniques_used = lift.get_techniques_used_by_all_groups()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1346"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(techniques_used)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'aliases': ['APT41'],\n",
       " 'type': 'intrusion-set',\n",
       " 'name': 'APT41',\n",
       " 'description': '[APT41](https://attack.mitre.org/groups/G0096) is a group that carries out Chinese state-sponsored espionage activity in addition to financially motivated activity. [APT41](https://attack.mitre.org/groups/G0096) has been active since as early as 2012. The group has been observed targeting healthcare, telecom, technology, and video game industries in 14 countries.(Citation: FireEye APT41 Aug 2019)',\n",
       " 'external_references': [{'external_id': 'G0096',\n",
       "   'source_name': 'mitre-attack',\n",
       "   'url': 'https://attack.mitre.org/groups/G0096'},\n",
       "  {'description': '(Citation: FireEye APT41 2019)', 'source_name': 'APT41'},\n",
       "  {'description': 'Fraser, N., et al. (2019, August 7). Double DragonAPT41, a dual espionage and cyber crime operation APT41. Retrieved September 23, 2019.',\n",
       "   'source_name': 'FireEye APT41 Aug 2019',\n",
       "   'url': 'https://content.fireeye.com/apt-41/rpt-apt41'}],\n",
       " 'created_by_ref': 'identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5',\n",
       " 'created': '2019-09-23T13:43:36.945Z',\n",
       " 'id': 'intrusion-set--18854f55-ac7c-4634-bd9a-352dd07613b7',\n",
       " 'modified': '2019-10-14T21:52:59.301Z',\n",
       " 'object_marking_refs': ['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'],\n",
       " 'x_mitre_version': '1.0',\n",
       " 'technique_ref': 'attack-pattern--3489cfc5-640f-4bb3-a103-9137b97de79f',\n",
       " 'relationship_description': ' [APT41](https://attack.mitre.org/groups/G0096) used the <code>net share</code> command as part of network reconnaissance.',\n",
       " 'relationship_id': 'relationship--4f6e677d-427b-4342-b35c-57f4f3ad4ff8',\n",
       " 'technique': 'Network Share Discovery',\n",
       " 'technique_description': 'Networks often contain shared network drives and folders that enable users to access file directories on various systems across a network. \\n\\n### Windows\\n\\nFile sharing over a Windows network occurs over the SMB protocol. (Citation: Wikipedia Shared Resource) (Citation: TechNet Shared Folder)\\n\\n[Net](https://attack.mitre.org/software/S0039) can be used to query a remote system for available shared drives using the <code>net view \\\\\\\\remotesystem</code> command. It can also be used to query shared drives on the local system using <code>net share</code>.\\n\\nAdversaries may look for folders and drives shared on remote systems as a means of identifying sources of information to gather as a precursor for Collection and to identify potential systems of interest for Lateral Movement.\\n\\n### Mac\\n\\nOn Mac, locally mounted shares can be viewed with the <code>df -aH</code> command.\\n\\n### Cloud\\n\\nCloud virtual networks may contain remote network shares or file storage services accessible to an adversary after they have obtained access to a system. For example, AWS, GCP, and Azure support creation of Network File System (NFS) shares and Server Message Block (SMB) shares that may be mapped on endpoint or cloud-based systems.(Citation: Amazon Creating an NFS File Share)(Citation: Google File servers on Compute Engine)',\n",
       " 'tactic': ['discovery'],\n",
       " 'technique_id': 'T1135',\n",
       " 'matrix': 'mitre-attack',\n",
       " 'platform': ['macOS', 'Windows', 'AWS', 'GCP', 'Azure'],\n",
       " 'data_sources': ['Process monitoring',\n",
       "  'Process command-line parameters',\n",
       "  'Network protocol analysis',\n",
       "  'Process use of network'],\n",
       " 'permissions_required': ['User']}"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "techniques_used[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create Navigator Group Layer Files - (Manual)\n",
    "-----------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a list of group dictionaries\n",
    "To make things easier, I create a list of dictionaries where each group name is the main key and the value is a list where I append every single technique involving that group. I get that information from the `get_techniques_used_by_all_groups()` results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "groups = lift.get_groups()\n",
    "groups = lift.remove_revoked(groups)\n",
    "groups_list = []\n",
    "for g in groups:\n",
    "    group_dict = dict()\n",
    "    group_dict[g['name']] = []\n",
    "    groups_list.append(group_dict)\n",
    "groups_list[89]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Group techniques by group\n",
    "We can then use the output of the `get_techniques_used_by_all_groups()` function and start appending techniques to the dictionaries with the key name that matches the group name being involved with each technique."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for group in groups_list:\n",
    "    for group_name,techniques_list in group.items():\n",
    "        for gut in techniques_used:\n",
    "            if group_name == gut['name']:\n",
    "                technique_dict = dict()\n",
    "                technique_dict['techniqueId'] = gut['technique_id']\n",
    "                technique_dict['techniqueName'] = gut['technique']\n",
    "                technique_dict['comment'] = gut['relationship_description']\n",
    "                technique_dict['tactic'] = gut['tactic']\n",
    "                technique_dict['group_id'] = gut['external_references'][0]['external_id']\n",
    "                techniques_list.append(technique_dict)\n",
    "groups_list[89]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run dynamic navigator layer template"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "for group in groups_list:\n",
    "    for k,v in group.items():\n",
    "        if v:\n",
    "            actor_layer = {\n",
    "                \"description\": (\"Enterprise techniques used by {0}, ATT&CK group {1} v1.0\".format(k,v[0]['group_id'])),\n",
    "                \"name\": (\"{0} ({1})\".format(k,v[0]['group_id'])),\n",
    "                \"domain\": \"mitre-enterprise\",\n",
    "                \"version\": \"2.2\",\n",
    "                \"techniques\": [\n",
    "                    {\n",
    "                        \"score\": 1,\n",
    "                        \"techniqueID\" : technique['techniqueId'],\n",
    "                        \"techniqueName\" : technique['techniqueName'],\n",
    "                        \"comment\": technique['comment']\n",
    "                    } for technique in v\n",
    "                ],\n",
    "                \"gradient\": {\n",
    "                    \"colors\": [\n",
    "                        \"#ffffff\",\n",
    "                        \"#ff6666\"\n",
    "                    ],\n",
    "                    \"minValue\": 0,\n",
    "                    \"maxValue\": 1\n",
    "                },\n",
    "                \"legendItems\": [\n",
    "                    {\n",
    "                        \"label\": (\"used by {}\".format(k)),\n",
    "                        \"color\": \"#ff6666\"\n",
    "                    }\n",
    "                ]\n",
    "            }\n",
    "            with open(('{0}_{1}.json'.format(k,v[0]['group_id'])), 'w') as f:\n",
    "                f.write(json.dumps(actor_layer))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! ls *.json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can delete all the files with the following command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! rm *.json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create Navigator Group Layer Files - (Automatic)\n",
    "-----------------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from attackcti import attack_client\n",
    "lift = attack_client()\n",
    "\n",
    "%time lift.export_groups_navigator_layers()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! ls *.json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can delete all the files with the following command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! rm *.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
